Endowing Virtual Characters with Expressive Conversational Skills

When humans interact with one another, socially intelligent conversational behaviors arise from the interaction of a number of different factors: the conversants’ personality, cultural knowledge, the ability to observe and reason about social relationships, and the ability to project and detect affective cues. For virtual agents to be socially intelligent, they must have an expressive conversational repertoire. Moreover, scientific progress in this area requires that these expressive capabilities be easily parameterized, to support experimentation, and that at least some of the factors mentioned above be used to control the parameters. In this talk, I describe our research on expressive spoken language generation, and discuss how our work aims for both psychological plausibility and realistic usability. To achieve psychological plausibility we build on theories and detailed studies of human language use, such as the Big Five theory of personality, and Brown and Levinsons theory of politeness [1,2,3]. To achieve realistic usability, we have developed both rule-based and trainable generation methods that can dynamically, and in real time, change an agents linguistic style by modifying the values of these theoretically motivated parameters. This will allow us to experiment with dynamically modifying an agent’s linguistic style based on theories of audience design, entrainment and alignment. We built the first generator based on B&L as part of the VIVA virtual theatre, an application for teaching English as a second language [1]. We recently extended these ideas in the PoLLy system [4], and evaluated human perceptions of the politeness variations that PoLLy can generate, and how these vary across cultures and discourse contexts, encoding relevant factors. See Table 1. Our work on personality is embodied in Personage one of the first parameterizable generators based on the Big Five theory, which provides 67 different parameters, controlling utterance length and polarity, lexical, syntactic and pragmatic choice, and rhetorical structure. See Table 2. Personage’s parameters are all motivated by previous corpus-based studies on the linguistic reflexes of personality. Our evaluation experiments show that humans recognize utterances as manifesting the personality that the agent intended.